Gaussian Assumption: the Least Favorable but the Most Useful

Gaussian assumption is the most well-known and widely used distribution in many fields such as engineering, statistics and physics. One of the major reasons why the Gaussian distribution has become so prominent is because of the Central Limit Theorem (CLT) and the fact that the distribution of noise in numerous engineering systems is well captured by the Gaussian distribution. Moreover, features such as analytical tractability and easy generation of other distributions from the Gaussian distribution contributed further to the popularity of Gaussian distribution. Especially, when there is no information about the distribution of observations, Gaussian assumption appears as the most conservative choice. This follows from the fact that the Gaussian distribution minimizes the Fisher information, which is the inverse of the CramérRao lower bound (CRLB) (or equivalently stated, the Gaussian distribution maximizes the CRLB). Therefore, any optimization based on the CRLB under the Gaussian assumption can be considered to be min-max optimal in the sense of minimizing the largest CRLB (see [1] and the references cited therein). Inspired by the early isoperimetric inequality for entropy introduced by Costa and Cover [2] and the more recent results of Rioul [3], Stoica and Babu [1], the goals of this paper are threefold: i) to illustrate a connection between [1] and the recent information theoretic results reported in [2], [3], ii) to present information theoretic and estimation theoretic justifications for the fact that the Gaussian assumption leads to the largest CRLB, iii) to show a slight extension of this result to the more general framework of correlated observations. Even though Stoica and Babu provided a simple and quite general proof of result that the largest CRLB is achievable by the Gaussian distribution, the proposed proof is only applicable to the situation when the observations are independent, i.e., the observation noise is white [1]. However, this result can be generalized to arbitrary correlations among samples. In many practical circumstances, the